On Yahoo Answers, Long Answers are Best

We provide an analysis of best answers (as chosen by questioners) on Yahoo Answers, a popular online Q&A site with millions of monthly contributors. Our analysis is done mainly through the lens of prediction: we compile a dataset that is as large and fine-grained as any considered before, generate features across a range of different classes, and build a classifier to predict which answers will be selected as “best”. On the dataset as a whole, despite the breadth and sophistication of our features and learning framework, we achieve virtually no performance edge over the following simple baseline: choose the longest answer. Propelled by this unexpected discovery, we perform a detailed analysis of answer length and how it relates to other variables of interest, such as answer time and number of answers. We explore subsets of the data designed to probe into areas where the longest-answer baseline may be handicapped, but we consistently find that it is competitive with our full-featured learner. Our results suggest future directions of study, e.g., controlled experimentation or user interviews, which may shed further light on why answer length is such a good proxy for (the questioner’s estimation of) answer quality.